fix(llm): prevent base llm flow truncation (#522) #659
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hey @Jacksunwei @hangfei |
AlankritVerma01
changed the title
This is invalid state for llm flow.
Jacksunwei
approved these changes
May 27, 2025
Jacksunwei
added
the
ready to pull
Jacksunwei
approved these changes
May 27, 2025
copybara-service bot
pushed a commit
that referenced
this pull request
May 28, 2025
…ted state and can happen when the model output limit is maxed out. Copybara import of the project: -- a6e3a22 by Alankrit Verma <[email protected]>: test(base_llm_flow): add test for infinite loop on truncated responses -- b5f2245 by Alankrit Verma <[email protected]>: feat(base_llm_flow): break run_async loop on partial/truncated events -- cbbae4c by Wei Sun (Jack) <[email protected]>: Raise ValueError if the last event is partial. This is invalid state for llm flow. -- 6eebae0 by Wei Sun (Jack) <[email protected]>: Delete tests/unittests/flows/llm_flows/test_base_llm_flow_truncation.py -- e08b0ab by Wei Sun (Jack) <[email protected]>: format base_llm_flow.py COPYBARA_INTEGRATE_REVIEW=#659 from AlankritVerma01:fix/522-base-llm-flow-truncation e08b0ab PiperOrigin-RevId: 764380150
raflFaisal
pushed a commit
to raflFaisal/adk-python
that referenced
this pull request
May 28, 2025
…ted state and can happen when the model output limit is maxed out. Copybara import of the project: -- a6e3a22 by Alankrit Verma <[email protected]>: test(base_llm_flow): add test for infinite loop on truncated responses -- b5f2245 by Alankrit Verma <[email protected]>: feat(base_llm_flow): break run_async loop on partial/truncated events -- cbbae4c by Wei Sun (Jack) <[email protected]>: Raise ValueError if the last event is partial. This is invalid state for llm flow. -- 6eebae0 by Wei Sun (Jack) <[email protected]>: Delete tests/unittests/flows/llm_flows/test_base_llm_flow_truncation.py -- e08b0ab by Wei Sun (Jack) <[email protected]>: format base_llm_flow.py COPYBARA_INTEGRATE_REVIEW=google#659 from AlankritVerma01:fix/522-base-llm-flow-truncation e08b0ab PiperOrigin-RevId: 764380150
|
Merged in 2a07eef |
whhlct
pushed a commit
to whhlct/adk-python
that referenced
this pull request
Jun 6, 2025
…ted state and can happen when the model output limit is maxed out. Copybara import of the project: -- a6e3a22 by Alankrit Verma <[email protected]>: test(base_llm_flow): add test for infinite loop on truncated responses -- b5f2245 by Alankrit Verma <[email protected]>: feat(base_llm_flow): break run_async loop on partial/truncated events -- cbbae4c by Wei Sun (Jack) <[email protected]>: Raise ValueError if the last event is partial. This is invalid state for llm flow. -- 6eebae0 by Wei Sun (Jack) <[email protected]>: Delete tests/unittests/flows/llm_flows/test_base_llm_flow_truncation.py -- e08b0ab by Wei Sun (Jack) <[email protected]>: format base_llm_flow.py COPYBARA_INTEGRATE_REVIEW=google#659 from AlankritVerma01:fix/522-base-llm-flow-truncation e08b0ab PiperOrigin-RevId: 764380150
coder-aditi
pushed a commit
to coder-aditi/adk-python
that referenced
this pull request
Jul 2, 2025
…ted state and can happen when the model output limit is maxed out. Copybara import of the project: -- a6e3a22 by Alankrit Verma <[email protected]>: test(base_llm_flow): add test for infinite loop on truncated responses -- b5f2245 by Alankrit Verma <[email protected]>: feat(base_llm_flow): break run_async loop on partial/truncated events -- cbbae4c by Wei Sun (Jack) <[email protected]>: Raise ValueError if the last event is partial. This is invalid state for llm flow. -- 6eebae0 by Wei Sun (Jack) <[email protected]>: Delete tests/unittests/flows/llm_flows/test_base_llm_flow_truncation.py -- e08b0ab by Wei Sun (Jack) <[email protected]>: format base_llm_flow.py COPYBARA_INTEGRATE_REVIEW=google#659 from AlankritVerma01:fix/522-base-llm-flow-truncation e08b0ab PiperOrigin-RevId: 764380150
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #522
Previously,
BaseLlmFlow.run_asyncwould loop indefinitely whenever the LLM response was truncated due to hitting itsmax_tokenslimit. The loop’s break condition only checked for “no events” orlast_event.is_final_response(), neither of which fires for a partial/truncated chunk—so the generator re-invoked_run_one_step_asyncendlessly.With this change, we:
or last_event.partialto the break condition inrun_async.partial=True),run_asyncyields it exactly once and then terminates, avoiding infinite loops.Test Plan
test_base_llm_flow_truncation.pyverifies that:run_asyncemits exactly one event and then stops.